
Image Analysis via Soft Computing: 

Prototype Applications at NASA KSC 
and Product Commercialization. 


NA 


Kennedy Space Center 


Jesus A. Dominguez 
Steve Klinko 



ASRC Aerospace Corp. 







photo: NASA/Pat P^IcCracken 


Outline 


Kennedy Space Center 



• System Development 

• Performance results compared with 
existing approaches. 

• NASA applications. 

• Commercialization. 
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Soft Computing (SC): 

differs from conventional (hard) computing in that, unlike hard 

computing, it is tolerant of imprecision, uncertainty, partial truth, and 
approximation. 

provides flexible information processing to handle real life ambiguous 
situations and achieve tractability, robustness, low solution cost, and close 
resemblance of_ human decision making . 
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System Development 

Kennedy Space Center 

• FEED. C Fuzzy. Reasoning. Edge Detection ): 

Image edge extraction technique developed at KSC (patent protected). 

• FRAT (Fuzzy Reasoning Adaptive Thresholding): 

Image binarization technique developed at KSC (patent protected). 

• Set oj Image Enhancen lent Technigu es: 

Techniques developed at KSC (one patent protected). 

• Visual/Pattern Recognitiorr 

Commercially available technique (NeuroShell) via Artificial Neural 
Network (ANN) and GA (Genetic Algorithm). 
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Software implementation (Cont). 
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Performance results compared with existing approaches: FRED 



Original Image: CD containing a hard-to- 
see major scratch on the center 


FRED: The major scratch is clearly 
shown as well as other minor ones 
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Performance results compared with existing approaches: FRED 



Sobel Approach: less clear features, major 
scratch invisible 




Prewit Approach: less clear feature 
major scratch invisible 
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Performance results compared with existing approaches: FRAT 
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Original 8-bit Image 
Size: 246x245 


Huang- Wang Method 
CPU time: 10.8 ms 
Threshold: 89 






Otsu’s Method 
CPU time: 1.5 ms 
Threshold: 88 
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Threshold: 8 
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NASA Applications 
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• Real-Time (RT) Anomaly Detection. 

• Real-Time (RT) Moving Debris Detection. 

• Columbia Investigation. 



RT Anomaly Detection 
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Image Preprocessing . 
Enhancement 

Segmentation (Binarization) 



• Classification and Learning Processes . 
Artificial Neural Network (ANN) 
Genetic Algorithm (GA). 
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Astronauts tjaining_ on the emergency egress system. 
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Anomalies on the basket slidewire 



Broken strand. Molten spots caused by lighting. 
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Cable and Line Inspection Mechanism (CLIM) 
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he National Aeronautics and Space Administration 
(NASA) seeks to transfer the NASA-<ievclopcd Cable 
and Line Inspection Mechanism technology to private 
industry for use in commercial applications. This mecha- 
nism was developed at the John F. Kennedy Space Center 
(KSC) to provide a means for automated inspection of 
the seven slidewire cables used in the emergency egress 
system for the Space Shuttle. There are two sets of 
gantry cables plus an overhead lightning cable that require 
periodic inspection. These cables are nonferrous stainless 
steel; therefore, magnetic cable testers are not suitable for 
such inspections. Prior to this invention, cable inspections 
required 150 man-hours twice per year, with inspectors 
being hoisted in baskets to manually inspect the cables by 


direct touch and sight. The CLIM technology eliminates 
the hazardous, manpower-intensive, and time-consuming 
methods previously required to maintain the emergency 
egress system at peak performance standards. In addition. 
CLIM is capable of inspecting the lop end of ferrous 
wire ropes near the attachment point in the cable housing 
where magnetic cable testers arc unable to reach. CUM 
has a further application with respect to radio frequency 
(RF) tower guy -wire inspections. The low-carbon, low- 
magnetic inductance of the stainless- steel guy-wire cables, 
combined with added RF radiation interference from the 
tower, yields magnetic cable testers ineffective. Therefore. 
CLIM’s ability to conduct a 360-degree view of the cable 
without incurring RF radiation interference is significant. 
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CUM at the lab. CLIM at the Shuttle Pad 



Cable & Line Inspection Mechanism (CLIM) built by NASA. 


Slidewire image acquired by CLIM 
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Anomaly Detection via ANN-GA engine (output) 
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RT Anomaly Detection (Cont.) 
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RT Anomaly Detection (Cont.) 
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End-user front 
end built via 
MFC. 


Training Stage. 
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NeuroShell Classifier - Trained Network Information 

Network filename: C:\sessions\data\session11\session11G3.net 

The network was trained on: 

Filename: C:\sessions\data\session1 1\session1 1 .dat 

Total data rows: 798 

Training rows: 798 

Start row: 1 

End row: 798 


ASRC Aerospace Corp. 




Results of training session: 

Training time: 1:20: *16" 

Generations trained: 178 

Correct classifications: 98.87% (789 of 798) 

Incorrect classifications: 1.13% (9 of 798) 
Performance by category: 

"O" 90.48% (38 of 42) 

”10" 99.34% (751 of 756) 


Network structure: 

Training strategy: Genetic 
Output name: "Class" 

Number of inputs: 13 

List of inputs and their relative importance: 
"ContrastO" 0.006 

"Contrastl" 0.163 


Development of 
classification 
model via ANN & 
GA (NeuroShell) 
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Anomaly Detected 
& displayed in RT. 


Firing stage 






RT Anomaly Detection (Cont.) 
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Automated Region of Interest (ROI) 
extraction and anomaly detection. 

Original image acquired by CUM 
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Blob/FOD Selection and Trajectory Computation 
Logical path analysis (consecutive images). 







RT Moving FOD Detection (Cont.) 
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Tail 

Tail Service Mask 
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Columbia Investigation 

Kennedy Space ‘Center __ 

Foam Debris 
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2D: Detection & Location . 

Segmentation (Binarization)- 
Characterization (center of mass, borders, etc.) 


3D: Location & Trajectory 

Optimal path at three consecutive 3D projections. 
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Columbia Investigation 
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STS-107 REPORT: 
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2D-Detection, 3D-Location & 3D-Velocity Estimation of 
Foam Debris Based on Images acquired by E212 & E208 

Video Cameras. 
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Jesus A. Dominguez, ASRC Aerospace Corp . 
NASA Kennedy Space Center, June 12, 2003 


Columbia Investigation: 


Foam Debris Detection/Location (Cont.) 
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Columbia Investigation: 

Foam Debris Detection/Location (Cont.) vw 
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Columbia Investigation: 

Foam Debris Detection/Location (Cont.) 
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E212 at 21.753 s ET208 at 21 .757 s E212 at 21.769 s 





Columbia Investigation: 

Foam Debris Detection/Location (Cont.) 


ASRC Aerospace Corp. 


Kennedy Space Center 



E212 at 21.784 s ET208 at 21.791 s E212 at 21.800 s 
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E212 at 21.784 s E212 at 21.800 s 

ET208 at 21.791 s Optimized path. 



Columbia Investigation: 

Foam Debris Detection/Location (Cont.) vk 
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Columbia Investigation: 
Foam Debris Trajectory 

Kennedy Space Center 



-150 -100 



Current Work Lane-Nelson work via LighWave3D 
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Columbia Investigation: 
Foam Debris Trajectory 

Kennedy Space Center 




Current Work Lane-Nelson work via LighWave3D 
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Columbia Investigation: 
Foam Debris Velocity 

Kennedy Space Center 


3D Velocity Estimation 
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Proposed Debris Analysis Software 
System Development at KSC 


Kennedy Space Center 


Automated Debris Detection at KSC VAB Launch Analysis Laboratory 




/in Windows 

Current OS housing FRAT, 
FRED, SC-based image 
pattern recognition. 



VAB Launch Analysis Laboratory at 
KSC equipped with recently acquired 
SGI Reality Center facility with a 7- 
foot display, and advanced SGI 
TP9500 data management 
subsystem. 
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Commercialization: Licensing 
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Patent 


NASA KSC 


Technology Marketing 

BRT1 

INTERHUIONAl 

Research Triangle Institute (RTI) 
Center for Technology Applications 
PO Box 12194, 3040 Cornwallis 
Research Triangle Park, NC 27709 


NASA Technology Applications Team: 


Kirsten Rieth 
Phone: (919) 967-4991 
Fax: (919)541-6221 

Email: krieth@rti.org 


John Geikler 

Phone: (919) 941-8372 

Fax: (919)941-8399 

Email: johng @ thesolutioncenter.com 


Commercialization (Cont.) 
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http://nasa. rti. org/ksc/imaging 
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NASA's Kennedy Space Center... 


The PerttoBe 
Register Interest 
ConUcts 

IS seeking companies to license its portfolio of mage processing technologies The 
technologies span the range from tow- to high-level computer vision capabilities and 
have significant synergistic potential, as NASA has demonstrated 

Commercial Applications 

The KSC image processing portfolio has potential for a broad range of applications 



• Medical diagnostic imaging and Computer-Aided Detection (CAD) 

• Remote sensing, including military and earth resources 

• Geophysical applications, including seismic, GPR, SAR, and magnetic anomaly 

• Biometncs, including finger, facial, and iris scans 

• Optical character recognition (OCR) and other scanning applications 

• Secunty, surveillance, and baggage inspection systems 

• Scanning and handwntmg analysis, including POA data entry systems 

• Industrial machine vision and inspection systems 


HHI 

Please find out more about NASA KSC's portfolio of imagrg techno* opes. 
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Binarization of Noisy Images 
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Catalog 
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International Joint Conference on Neural Networks 
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Commercialization: Status Summary 
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• Patents already filed by NASA (February 2004). 

• Marketing and Promotion already in place by RTI. 

• 2 Software Usage Agreements executed. 

• 2 License Agreements executed. 

• 3 License Agreements being executed. 

• 3 License Agreements in negotiation. 


Commercialization: Selected Applications 
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Image enhancement 
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Binarization via FRAT. 
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Enhancement prior Binarization 
(via FRAT). 


Binarization via Otsu method. 
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Binarization via Huang- Wang method. 
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Image enhancement 
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AS&E Inc. 

Human Screening & Privacy Protection 
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NASA JSC 

Visual inspection based on x-ray images 
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Original x-ray image of 
two pyrotechnic valves. 







Commercialization: Selected Applications 
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Bay Ion School of Medicine 

3D heart Visualization 
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Original image 
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FRAT 




A. Cross-sectional (tomographic) image of the left ventricular cavity of the dog heart 
acquired by an intracardiac echocardiography (ICE) catheter. Circle indicates ICE catheter. 
Dark region indicates blood-filled cavity, which is bounded by the interior surface of the 
heart (endocardium). B. Result of FRED demo when applied to the ICE image depicted in A. 
C. Result of FRAT demo when applied to the ICE image depicted in A. 
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Zeus Technologies, Inc. 
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Zeus Technologies, Inc. 


X-ray image visualization enhancement 
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